Timing pulses of pulsars has proved to be a most powerful technique useful to a host of research areas in astronomy and physics. Importantly, the precision of this timing is not only affected by radiometer noise, but also by intrinsic pulse shape changes, interstellar medium (ISM) evolution, instrumental distortions, etc. In this paper we review the known causes of pulse shape variations and assess their effect on the precision and accuracy of a single measurement of pulse arrival time with current instrumentation. Throughout this analysis we use the brightest and most precisely timed millisecond pulsar (MSP), PSR J0437-4715, as a case study, and develop a set of diagnostic tools to evaluate profile stability in timing observations. We conclude that most causes of distortion can be either corrected by state-of-the-art techniques or taken into account in the estimation of time-of-arrival (TOA) uncertainties. The advent of a new generation of radio telescopes (e.g. the Square Kilometre Array, SKA), and their increase in collecting area has sparked speculation about the timing precision achievable through increases in gain. Based on our analysis of current data, we predict that for normal-brightness MSPs a TOA precision of between 80 and 230 ns can be achieved at 1.4 GHz with 10-minute integrations by the SKA. The actual rms timing residuals for each pulsar will be approximately at the same level only if all the other influences on timing precision (e.g. ISM, spin noise) are either corrected, modelled, or negligible.